Kierunek: Informatyka - Data Science
Autor rozwiÄ
zaĆ: BartĆomiej JamioĆkowski
Biblioteka LIME: https://github.com/marcotcr/lime (Dokumentacja API: https://lime-ml.readthedocs.io/en/latest/)
NiezbÄdne pakiety i moduĆy na potrzeby wprowadzenia
import json
from functools import partial
import matplotlib.pyplot as plt
import numpy as np
import torch
import random
import cv2
from lime import lime_image
from PIL import Image
from skimage.segmentation import mark_boundaries
from torchvision import models, transforms
Funkcja do wczytywania wskazanego obrazka oraz konwersji do palety RGB.
def get_image(path):
with open(path, 'rb') as f:
with Image.open(f) as img:
return img.convert('RGB')
Funkcja do przeksztaĆcania obrazka (zwrĂłconego przez funkcjÄ get_image) w tensor, akceptowalny na wejĆciu sieci neronowej.
def image_to_tensor(img):
transformer = transforms.Compose([
transforms.Resize((256, 256)),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(
mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
])
return transformer(img).unsqueeze(0)
Funkcja uĆŒywana przez LIME, przyjmuje na wejĆciu zbiĂłr obrazkĂłw, a zwraca prawdopodobieĆstwa klas. NaleĆŒy jÄ
przekazaÄ do lime_image.LimeImageExplainer().explain_instance przy uĆŒyciu partial, jako partial(predict_batch, <model>), gdzie modelem w naszym wypadku bÄdÄ
sieci neuronowe. PrzykĆady uĆŒycia sÄ
zawarte w tym notebooku.
def predict_batch(model, images):
model.eval()
transformer = transforms.Compose([
transforms.ToTensor(),
transforms.Normalize(
mean=[0.485, 0.456, 0.406],
std=[0.229, 0.224, 0.225])
])
model.eval()
batch = torch.stack(tuple(transformer(i) for i in images), dim=0)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
batch = batch.to(device)
logits = model(batch)
probas = torch.nn.functional.softmax(logits, dim=1)
return probas.detach().cpu().numpy()
Funkcja, ktĂłra przeksztaĆca obrazek w format akceptowany na wejĆciu przez LIME. PrzykĆady uĆŒycia sÄ zawarte w tym notebooku.
def lime_transformer(image):
transformer = transforms.Compose([
transforms.Resize((256, 256)),
transforms.CenterCrop(224)
])
return np.array(transformer(image))
LIME jest gĆĂłwnie wykorzystywane do wyjaĆniania predykcji tzw. czarnych skrzynek, czyli modeli nieinterpretowalnych. Idealnymi kandydatami sÄ GĆÄbokie Sieci Neuronowe, dlatego sprĂłbujemy wyjaĆniÄ niektĂłre predykcje gotowych modeli.
Plik ./data/imagenet_class_index.json zawiera przypisanie klas obrazkĂłw do indeksĂłw. Jest to istotne, poniewaĆŒ zwracane wyniki (np. wartoĆci funkcji logit na wyjĆciu sieci neuronowych) wykorzystujÄ
to, zwracajÄ
c wyniki w zadanej kolejnoĆci.
with open("./data/imagenet_class_index.json") as f:
content = json.load(f)
index_to_label = {
int(index): data[1]
for index, data in content.items()
}
image_to_classify = get_image("./data/dogs.png")
plt.imshow(image_to_classify)
<matplotlib.image.AxesImage at 0x1eefa000ec0>
img_tensor = image_to_tensor(image_to_classify)
inception_v3 = models.inception_v3(pretrained=True)
C:\Users\Bartek\anaconda3\Lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( C:\Users\Bartek\anaconda3\Lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=Inception_V3_Weights.IMAGENET1K_V1`. You can also use `weights=Inception_V3_Weights.DEFAULT` to get the most up-to-date weights. warnings.warn(msg)
inception_v3.eval()
logits = inception_v3(img_tensor)
ZwrĂłÄ uwagÄ, ĆŒe model zwraca wartoĆci funkcji logit, a nie prawdopodobieĆstwa klas, dlatego wyniki trzeba przetworzyÄ (np. przy uĆŒyciu funkcji softmax).
probas = torch.nn.functional.softmax(logits, dim=1)
SprawdĆșmy N najbardziej prawdopodobnych klas
TOP_N_LABELS = 15
probas_top = probas.topk(TOP_N_LABELS)
top_probas = probas_top[0][0].detach().numpy()
top_labels = probas_top[1][0].detach().numpy()
for proba, label in zip(top_probas, top_labels):
print(f"Class: {index_to_label[label]:<30} | Probability: {proba:.6f}")
Class: Bernese_mountain_dog | Probability: 0.935930 Class: EntleBucher | Probability: 0.038448 Class: Appenzeller | Probability: 0.023756 Class: Greater_Swiss_Mountain_dog | Probability: 0.001818 Class: Gordon_setter | Probability: 0.000009 Class: Blenheim_spaniel | Probability: 0.000007 Class: English_springer | Probability: 0.000002 Class: tabby | Probability: 0.000002 Class: robin | Probability: 0.000001 Class: guinea_pig | Probability: 0.000001 Class: amphibian | Probability: 0.000001 Class: Japanese_spaniel | Probability: 0.000001 Class: African_grey | Probability: 0.000001 Class: Brittany_spaniel | Probability: 0.000001 Class: toucan | Probability: 0.000001
def get_prediction_probabilities(image, model):
img_tensor = image_to_tensor(image)
model.eval()
logits = model(img_tensor)
probas = torch.nn.functional.softmax(logits, dim=1)
TOP_N_LABELS = 15
probas_top = probas.topk(TOP_N_LABELS)
top_probas = probas_top[0][0].detach().numpy()
top_labels = probas_top[1][0].detach().numpy()
for proba, label in zip(top_probas, top_labels):
print(f"Class: {index_to_label[label]:<30} | Probability: {proba:.6f}")
exercise_image = get_image("./data/cat_mouse.jpeg")
plt.imshow(exercise_image)
<matplotlib.image.AxesImage at 0x1eefb33c080>
get_prediction_probabilities(exercise_image, inception_v3)
Class: Egyptian_cat | Probability: 0.967491 Class: tabby | Probability: 0.024167 Class: lynx | Probability: 0.005490 Class: tiger_cat | Probability: 0.002165 Class: Persian_cat | Probability: 0.000105 Class: Angora | Probability: 0.000074 Class: swab | Probability: 0.000071 Class: Madagascar_cat | Probability: 0.000064 Class: snow_leopard | Probability: 0.000040 Class: tile_roof | Probability: 0.000037 Class: indri | Probability: 0.000020 Class: leopard | Probability: 0.000016 Class: Siamese_cat | Probability: 0.000011 Class: ram | Probability: 0.000010 Class: crate | Probability: 0.000009
Z uzyskanych wynikĂłw wynika, ĆŒe najbardziej prawdopodobnÄ klasÄ jest 'Egyptian_cat'.
Chcemy wiedzieÄ dlaczego klasa Bernese_mountain_dog zostaĆa uznana przez sieÄ neuronowÄ
za najbardziej prawdopodobnÄ
(to znaczy - ktĂłre piksele obrazka o tym zadecydowaĆy). W tym celu wĆaĆnie wykorzystamy LIME.
W jaki sposĂłb dziaĆa LIME na obrazkach?
explainer = lime_image.LimeImageExplainer()
explanation = explainer.explain_instance(
image=lime_transformer(image_to_classify),
classifier_fn=partial(predict_batch, inception_v3),
top_labels=5,
num_samples=1000)
0%| | 0/1000 [00:00<?, ?it/s]
MajÄ c te dane moĆŒemy teraz sprawdziÄ ktĂłre kategorie sÄ najbardziej prawdopodobne
for index in explanation.top_labels:
print(index_to_label[index])
Bernese_mountain_dog EntleBucher Appenzeller Greater_Swiss_Mountain_dog Gordon_setter
Zobaczmy co wpĆynÄĆo na wybranie Bernese_mountain_dog jako najbardziej prawdopodobnej klasy.
image, mask = explanation.get_image_and_mask(
label=explanation.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x1eefb155970>
NUM_FEATURES najlepiej zmieniaÄ w zakresie 1:50
NUM_FEATURES = 40 # 10
image, mask = explanation.get_image_and_mask(
label=explanation.top_labels[0],
positive_only=False,
negative_only=False,
num_features=NUM_FEATURES,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x1ee80cab9e0>
ZwiÄkszajÄ c wartoĆÄ parametru 'NUM_FEATURES' moĆŒna zauwaĆŒyÄ, ĆŒe do pewnego momentu zwiÄksza siÄ liczba zielonych i czerwonych fragmentĂłw obrazu. Oznacza to, ĆŒe model bierze pod uwagÄ coraz wiÄcej regionĂłw obrazu. Po przekroczeniu pewnej wartoĆci dodawanych jest coraz mniej istotnych superpikseli, co skutkuje mniej interpretowalnymi wynikami.
Zielone fragmenty oznaczajÄ "superpiksele", ktĂłre pozytywnie wpĆywajÄ na predykowanÄ klasÄ. Czerwone fragmenty wpĆywajÄ negatywnie.
Superpiksel to grupa sÄ siadujÄ cych pikseli na obrazie, ktĂłre sÄ pogrupowane w wiÄksze jednostki ze wzglÄdu na ich podobne wĆaĆciwoĆci. PozwalajÄ one na zmniejszenie liczby jednostek do analizy obrazu przy jednoczesnym zachowaniu istotnych informacji.
Jeden superpiksel ma odzwierciedlenie w jednym pikselu z obrazka, pod warunkiem ĆŒe wspomniany piksel naleĆŒy do grupy pikseli tworzÄ cych dany superpiksel. Na poziomie analizy obrazu dziaĆa on tak, jakby byĆ odpowiednikiem jednego punktu danych, ktĂłry wpĆywa na decyzjÄ modelu.
Zobaczmy jak to siÄ prezentuje dla drugiej najbardziej prawdopodobnej klasy, czyli EntleBucher, ktĂłra jednak otrzymaĆa jedyne 3.8%.
image, mask = explanation.get_image_and_mask(
label=explanation.top_labels[1],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x1eefb1bd7c0>
UstawiajÄ
c wartoĆci hide_rest oraz positive_only na True jesteĆmy w stanie zostawiÄ tylko te piksele, ktĂłre potwierdzaĆy przynaleĆŒnoĆÄ do danej klasy
Musimy jednak pamiÄtaÄ o przeskalowaniu rezultatu przy pomocy (boundaries).astype(np.uint8)
image, mask = explanation.get_image_and_mask(
label=explanation.top_labels[0],
positive_only=True,
negative_only=False,
num_features=10,
hide_rest=True)
boundaries = mark_boundaries(image, mask)
plt.imshow((boundaries).astype(np.uint8))
<matplotlib.image.AxesImage at 0x1eefb181640>
MoĆŒemy rĂłwnieĆŒ zostawiÄ tylko te piksele, ktĂłre zaprzeczaĆy przynaleĆŒnoĆci do danej klasy
image, mask = explanation.get_image_and_mask(
label=explanation.top_labels[0],
positive_only=False,
negative_only=True,
num_features=10,
hide_rest=True)
boundaries = mark_boundaries(image, mask)
cropped_image_ndarray = (boundaries).astype(np.uint8)
plt.imshow(cropped_image_ndarray)
<matplotlib.image.AxesImage at 0x1ee80dfe6f0>
A nastÄpnie sprawdziÄÂ co model sÄ dzi o tak wyciÄtym obrazku
cropped_image_pil = Image.fromarray(cropped_image_ndarray)
get_prediction_probabilities(cropped_image_pil, inception_v3)
Class: African_grey | Probability: 0.968853 Class: sulphur-crested_cockatoo | Probability: 0.014763 Class: West_Highland_white_terrier | Probability: 0.003940 Class: toy_poodle | Probability: 0.002781 Class: miniature_poodle | Probability: 0.002558 Class: feather_boa | Probability: 0.001530 Class: teddy | Probability: 0.000714 Class: Sealyham_terrier | Probability: 0.000709 Class: komondor | Probability: 0.000457 Class: standard_poodle | Probability: 0.000301 Class: vulture | Probability: 0.000265 Class: Lhasa | Probability: 0.000248 Class: hen | Probability: 0.000246 Class: Bedlington_terrier | Probability: 0.000201 Class: Persian_cat | Probability: 0.000182
I jak go teraz widzi model
cropped_image_explanation = explainer.explain_instance(
image=lime_transformer(cropped_image_pil),
classifier_fn=partial(predict_batch, inception_v3),
top_labels=5,
num_samples=1000)
0%| | 0/1000 [00:00<?, ?it/s]
image, mask = cropped_image_explanation.get_image_and_mask(
label=cropped_image_explanation.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x1eefb33e180>
Przetestujmy dziaĆanie na innym modelu - AlexNet
alexnet = models.alexnet(pretrained=True)
C:\Users\Bartek\anaconda3\Lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( C:\Users\Bartek\anaconda3\Lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=AlexNet_Weights.IMAGENET1K_V1`. You can also use `weights=AlexNet_Weights.DEFAULT` to get the most up-to-date weights. warnings.warn(msg)
explanation_alexnet = explainer.explain_instance(
image=lime_transformer(image_to_classify),
classifier_fn=partial(predict_batch, alexnet),
top_labels=5,
num_samples=1000)
0%| | 0/1000 [00:00<?, ?it/s]
for index_alex, index_inception in zip(explanation_alexnet.top_labels, explanation.top_labels):
print(f"{index_to_label[index_alex]:30} | {index_to_label[index_inception]:30}")
Bernese_mountain_dog | Bernese_mountain_dog EntleBucher | EntleBucher Greater_Swiss_Mountain_dog | Appenzeller Appenzeller | Greater_Swiss_Mountain_dog basset | Gordon_setter
Jak widaÄ, klasy nieco siÄ rĂłĆŒniÄ , ale TOP 1 pozostaje takie samo.
image, mask = explanation_alexnet.get_image_and_mask(
label=explanation_alexnet.top_labels[0],
positive_only=False,
negative_only=False,
num_features=10,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
<matplotlib.image.AxesImage at 0x1ee80e4a3c0>
WyjaĆnienie dla AlexNet jak siÄ moĆŒna byĆo spodziewaÄ - teĆŒÂ siÄ rĂłĆŒni, jednak w dalszym ciÄ
gu do klasyfikacji psa istotny jest... pies :)
print("inception_v3")
get_prediction_probabilities(image_to_classify, inception_v3)
print()
print("alexnet")
get_prediction_probabilities(image_to_classify, alexnet)
inception_v3 Class: Bernese_mountain_dog | Probability: 0.935930 Class: EntleBucher | Probability: 0.038448 Class: Appenzeller | Probability: 0.023756 Class: Greater_Swiss_Mountain_dog | Probability: 0.001818 Class: Gordon_setter | Probability: 0.000009 Class: Blenheim_spaniel | Probability: 0.000007 Class: English_springer | Probability: 0.000002 Class: tabby | Probability: 0.000002 Class: robin | Probability: 0.000001 Class: guinea_pig | Probability: 0.000001 Class: amphibian | Probability: 0.000001 Class: Japanese_spaniel | Probability: 0.000001 Class: African_grey | Probability: 0.000001 Class: Brittany_spaniel | Probability: 0.000001 Class: toucan | Probability: 0.000001 alexnet Class: Bernese_mountain_dog | Probability: 0.400542 Class: EntleBucher | Probability: 0.208727 Class: Greater_Swiss_Mountain_dog | Probability: 0.194558 Class: Appenzeller | Probability: 0.186891 Class: basset | Probability: 0.002400 Class: Cardigan | Probability: 0.002184 Class: Border_collie | Probability: 0.001509 Class: kelpie | Probability: 0.000398 Class: bluetick | Probability: 0.000314 Class: Blenheim_spaniel | Probability: 0.000276 Class: collie | Probability: 0.000255 Class: borzoi | Probability: 0.000188 Class: Walker_hound | Probability: 0.000177 Class: Tibetan_mastiff | Probability: 0.000171 Class: English_springer | Probability: 0.000156
W przypadku obu modeli najwyĆŒej ocenianÄ klasÄ jest 'Bernese_mountain_dog'. RĂłĆŒnica tutaj polega na tym, ĆŒe model inception_v3 jest bardziej pewny swojej predykcji (prawdopodobieĆstwo okoĆo 94%), podczas gdy model alexnet jest mniej pewny swojej predykcji (prawdopodobieĆstwo okoĆo 40%).
W folderze data znajduje siÄ zdjÄcie tukana. Zgodnie z poleceniem zostaĆ wybrany wĆasny obraz (inny niĆŒ amfibia) z dowolnej klasy rozpoznawanej przez sieÄ. Wykorzystano obraz spoza zbioru ImageNet, na ktĂłrym wytrenowana byĆa sieÄ. Obraz pochodzi z internetowej darmowej galerii Pexels, gdzie twĂłrcy udostÄpniajÄ
swoje zdjÄcia, obrazy oraz filmy bez opĆat licencyjnych. Zdjecie zostaĆo wykonane w 2022 roku, czyli w czasie kiedy istniaĆ juĆŒ zbiĂłr ImageNet. ĆčrĂłdĆo obrazu: https://www.pexels.com/photo/close-up-photo-of-bird-perched-on-tree-branch-11398673/

bird = get_image("./data/toucan.jpg")
inception_v3 = models.inception_v3(pretrained=True)
explanation_bird_inception_v3 = explainer.explain_instance(
image=lime_transformer(bird),
classifier_fn=partial(predict_batch, inception_v3),
top_labels=5,
num_samples=1000)
image, mask = explanation_bird_inception_v3.get_image_and_mask(
label=explanation_bird_inception_v3.top_labels[0],
positive_only=False,
negative_only=False,
num_features=8,
hide_rest=False)
boundaries = mark_boundaries(image, mask)
plt.imshow(boundaries)
C:\Users\Bartek\anaconda3\Lib\site-packages\torchvision\models\_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead. warnings.warn( C:\Users\Bartek\anaconda3\Lib\site-packages\torchvision\models\_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=Inception_V3_Weights.IMAGENET1K_V1`. You can also use `weights=Inception_V3_Weights.DEFAULT` to get the most up-to-date weights. warnings.warn(msg)
0%| | 0/1000 [00:00<?, ?it/s]
<matplotlib.image.AxesImage at 0x1ee8251e990>
Model inception_v3 jak i jego wyjaĆnienie rzeczywiĆcie sugerujÄ
tukana jako najbardziej prawdopodobnÄ
klasÄ:
for index in explanation_bird_inception_v3.top_labels:
print(index_to_label[index])
toucan hornbill school_bus king_penguin screwdriver
UĆŒyj dwĂłch rĂłĆŒnych sieci neuronowych (poza inception_v3, ktĂłrego przykĆad jest powyĆŒej) do wygenerowania wyjaĆnieĆ.
(skorzystaj z moduĆu torchvision: https://pytorch.org/vision/stable/models.html)
ResNet50
resnet50 = models.resnet50(weights='IMAGENET1K_V2')
explanation_bird_resnet50 = explainer.explain_instance(
image=lime_transformer(bird),
classifier_fn=partial(predict_batch, resnet50),
top_labels=5,
num_samples=1000)
image_resnet50, mask_resnet50 = explanation_bird_resnet50.get_image_and_mask(
label=explanation_bird_resnet50.top_labels[0],
positive_only=False,
negative_only=False,
num_features=8,
hide_rest=False)
boundaries_resnet50 = mark_boundaries(image_resnet50, mask_resnet50)
plt.imshow(boundaries_resnet50)
0%| | 0/1000 [00:00<?, ?it/s]
<matplotlib.image.AxesImage at 0x1eefb156180>
for index in explanation_bird_resnet50.top_labels:
print(index_to_label[index])
toucan hornbill magpie china_cabinet coucal
Model resnet50 jak i jego wyjaĆnienie rzeczywiĆcie sugerujÄ tukana jako najbardziej prawdopodobnÄ klasÄ.
DenseNet161
densenet161 = models.densenet161(weights='IMAGENET1K_V1')
explanation_bird_densenet161 = explainer.explain_instance(
image=lime_transformer(bird),
classifier_fn=partial(predict_batch, densenet161),
top_labels=5,
num_samples=1000)
image_densenet161, mask_densenet161 = explanation_bird_densenet161.get_image_and_mask(
label=explanation_bird_densenet161.top_labels[0],
positive_only=False,
negative_only=False,
num_features=8,
hide_rest=False)
boundaries_densenet161 = mark_boundaries(image_densenet161, mask_densenet161)
plt.imshow(boundaries_densenet161)
0%| | 0/1000 [00:00<?, ?it/s]
<matplotlib.image.AxesImage at 0x1eefb248800>
for index in explanation_bird_densenet161.top_labels:
print(index_to_label[index])
toucan hornbill bee_eater king_penguin macaw
Model densenet161 jak i jego wyjaĆnienie rzeczywiĆcie sugerujÄ tukana jako najbardziej prawdopodobnÄ klasÄ.
Zmodyfikuj oryginalny obrazek w taki sposĂłb, ĆŒeby najbardziej prawdopodobnÄ
klasÄ
dla kaĆŒdej z tych sieci nie byĆ tukan a jakiĆ inny ptak. W tym celu moĆŒesz "zasĆoniÄ" czarnym kwadratem (wartoĆÄ 0 w macierzy reprezentujÄ
cej obraz) obszary istotne przy klasyfikacji.
PrzydatnÄ
rzeczÄ
bÄdzie skorzystanie z opcji hide_rest w funkcji get_image_and_mask i pĂłĆșniejsza obrĂłbka obrazu
ResNet50
image_resnet50, mask_resnet50 = explanation_bird_resnet50.get_image_and_mask(
label=explanation_bird_resnet50.top_labels[0],
positive_only=True,
negative_only=False,
num_features=4,
hide_rest=False)
modified_image_resnet50 = image_resnet50.copy()
modified_image_resnet50[mask_resnet50 == 1] = 0
boundaries_resnet50 = mark_boundaries(modified_image_resnet50, mask_resnet50)
plt.imshow(boundaries_resnet50)
<matplotlib.image.AxesImage at 0x1eefb154b90>
modified_image_resnet50 = Image.fromarray(modified_image_resnet50.astype(np.uint8))
print("resnet50")
get_prediction_probabilities(modified_image_resnet50, resnet50)
resnet50 Class: European_gallinule | Probability: 0.101171 Class: hornbill | Probability: 0.097626 Class: toucan | Probability: 0.072013 Class: vulture | Probability: 0.021064 Class: house_finch | Probability: 0.018672 Class: goldfinch | Probability: 0.017910 Class: magpie | Probability: 0.016234 Class: lorikeet | Probability: 0.015110 Class: hummingbird | Probability: 0.012519 Class: black_stork | Probability: 0.010743 Class: coucal | Probability: 0.010647 Class: macaw | Probability: 0.009183 Class: black_grouse | Probability: 0.009128 Class: bulbul | Probability: 0.008470 Class: jacamar | Probability: 0.006364
Model resnet50 jak i jego wyjaĆnienie, po modyfikacji obrazu, sugerujÄ European_gallinule jako najbardziej prawdopodobnÄ klasÄ.
DenseNet
image_densenet161, mask_densenet161 = explanation_bird_densenet161.get_image_and_mask(
label=explanation_bird_densenet161.top_labels[0],
positive_only=True,
negative_only=False,
num_features=6,
hide_rest=False)
modified_image_densenet161 = image_resnet50.copy()
modified_image_densenet161[mask_densenet161 == 1] = 0
boundaries_densenet161 = mark_boundaries(modified_image_densenet161, mask_densenet161)
plt.imshow(boundaries_densenet161)
<matplotlib.image.AxesImage at 0x1ee8cba33b0>
modified_image_densenet161 = Image.fromarray(modified_image_densenet161.astype(np.uint8))
print("densenet161")
get_prediction_probabilities(modified_image_densenet161, densenet161)
densenet161 Class: coucal | Probability: 0.259482 Class: toucan | Probability: 0.201190 Class: quill | Probability: 0.080150 Class: magpie | Probability: 0.075107 Class: hornbill | Probability: 0.059314 Class: limpkin | Probability: 0.055361 Class: vulture | Probability: 0.045030 Class: black_stork | Probability: 0.044994 Class: kite | Probability: 0.031459 Class: European_gallinule | Probability: 0.014090 Class: bald_eagle | Probability: 0.012607 Class: hummingbird | Probability: 0.012000 Class: bittern | Probability: 0.011722 Class: little_blue_heron | Probability: 0.011524 Class: jacamar | Probability: 0.004920
Model densenet161 jak i jego wyjaĆnienie, po modyfikacji obrazu, sugerujÄ coucal jako najbardziej prawdopodobnÄ klasÄ.
Ponownie zmodyfikuj oryginalny obraz, ale tym razem zaszumiajÄ c go w losowy sposĂłb (przykĆadowa implementacja: https://www.geeksforgeeks.org/add-a-salt-and-pepper-noise-to-an-image-with-python/). Czy najbardziej prawdopodobna klasa zmienia siÄ wraz ze zmianÄ szumu? Przetestuj dla kaĆŒdego z modeli.
def add_noise(img):
row, col, _ = img.shape
number_of_pixels = random.randint(1000, 400000)
for i in range(number_of_pixels):
y_coord = random.randint(0, row - 1)
x_coord = random.randint(0, col - 1)
img[y_coord, x_coord] = 255
number_of_pixels = random.randint(1000, 400000)
for i in range(number_of_pixels):
y_coord = random.randint(0, row - 1)
x_coord = random.randint(0, col - 1)
img[y_coord, x_coord] = 0
return img
modified_bird = cv2.imread("./data/toucan.jpg", cv2.IMREAD_GRAYSCALE)
modified_bird = cv2.cvtColor(modified_bird, cv2.COLOR_BGR2RGB)
modified_bird = add_noise(modified_bird)
plt.imshow(modified_bird)
<matplotlib.image.AxesImage at 0x1ee8b95b890>
modified_bird = Image.fromarray(modified_bird.astype(np.uint8))
print("inception_v3")
get_prediction_probabilities(modified_bird, inception_v3)
print()
print("resnet50")
get_prediction_probabilities(modified_bird, resnet50)
print()
print("densenet161")
get_prediction_probabilities(modified_bird, densenet161)
inception_v3 Class: quail | Probability: 0.776598 Class: weevil | Probability: 0.020417 Class: toucan | Probability: 0.007665 Class: African_grey | Probability: 0.004659 Class: macaw | Probability: 0.004246 Class: robin | Probability: 0.004097 Class: vulture | Probability: 0.003831 Class: peacock | Probability: 0.003604 Class: indigo_bunting | Probability: 0.003570 Class: timber_wolf | Probability: 0.003233 Class: kite | Probability: 0.002840 Class: black_stork | Probability: 0.002763 Class: African_chameleon | Probability: 0.002687 Class: bald_eagle | Probability: 0.002518 Class: ruffed_grouse | Probability: 0.002475 resnet50 Class: quail | Probability: 0.139694 Class: coucal | Probability: 0.099183 Class: limpkin | Probability: 0.086909 Class: ruffed_grouse | Probability: 0.044887 Class: ptarmigan | Probability: 0.018959 Class: kite | Probability: 0.018371 Class: toucan | Probability: 0.017036 Class: black_grouse | Probability: 0.012643 Class: vulture | Probability: 0.011939 Class: partridge | Probability: 0.010224 Class: magpie | Probability: 0.008268 Class: howler_monkey | Probability: 0.008167 Class: bittern | Probability: 0.007386 Class: robin | Probability: 0.007237 Class: three-toed_sloth | Probability: 0.007137 densenet161 Class: limpkin | Probability: 0.061517 Class: quail | Probability: 0.055912 Class: airship | Probability: 0.045662 Class: vulture | Probability: 0.044176 Class: bittern | Probability: 0.034145 Class: kite | Probability: 0.030196 Class: military_uniform | Probability: 0.027608 Class: tiger_shark | Probability: 0.026835 Class: chainlink_fence | Probability: 0.024917 Class: tank | Probability: 0.024767 Class: junco | Probability: 0.022861 Class: crayfish | Probability: 0.019788 Class: parachute | Probability: 0.016637 Class: ibex | Probability: 0.015993 Class: bow | Probability: 0.015890
W zadaniu testowano 3 modele tj.: inception_v3, resnet50 oraz densenet161. We wszystkich przypadkach, przy zmianie szumu, najbardziej prawdopodobna klasa ulegaĆa zmianie.